feat: Render Markdown in chat and query output using rich by KylinMountain · Pull Request #22 · VectifyAI/OpenKB

KylinMountain · 2026-04-12T01:38:39Z

Summary

Use rich.live.Live + rich.markdown.Markdown to progressively render LLM responses with proper terminal formatting (headings, bold, code blocks, lists, etc.)
Applied to both openkb chat (interactive REPL) and openkb query (single-shot)
Falls back to plain text when --no-color is set (chat) or stdout is not a tty (query)
Add rich>=13.0 as an explicit dependency

Test plan

python -m pytest tests/ — 197 passed
openkb chat — verify headings, bold, code blocks render with formatting
openkb chat --no-color — verify plain text fallback
openkb query "question" — verify formatted output
openkb query "question" > file.txt — verify no ANSI codes in file

KylinMountain · 2026-04-12T01:40:06Z

Use rich's Live + Markdown to progressively render LLM responses in the chat REPL with proper formatting (headings, bold, code blocks, lists, etc.) instead of raw text output. Falls back to plain text when --no-color is set. Tool call lines still use prompt_toolkit styled output.

Apply the same rich Live + Markdown streaming render to `openkb query` as was added to chat. Falls back to plain text when stdout is not a tty.

KylinMountain · 2026-04-12T01:44:21Z

Code review

Found 1 issue:

In query.py, after live.stop() writes the tool call line to stdout, live.start() re-activates the auto-refresh thread (4 Hz by default). On the next refresh cycle, Live uses position_cursor() — which emits ANSI CURSOR_UP sequences based on the previous render height — to move back up into the already-printed area and overwrite it with the last Markdown renderable. This means the tool call text printed between live.stop() and live.start() will be overwritten/erased by the next Live refresh. The fix is to not re-start Live after a tool call (keep it stopped until more text arrives, then start a fresh Live), or to use live.console.print(tool_call_text) before stopping so it is incorporated into the live area.

OpenKB/openkb/agent/query.py

Lines 153 to 160 in 5d2b4b4

    
               args = getattr(raw, "arguments", "{}") 
        
               if live: 
        
                   live.stop() 
        
               sys.stdout.write(f"\n[tool call] {raw.name}({args})\n\n") 
        
               sys.stdout.flush() 
        
               if live: 
        
                   live.start() 
        
           elif item.type == "tool_call_output_item":

🤖 Generated with Claude Code

_{- If this code review was useful, please react with 👍. Otherwise, react with 👎.}

KylinMountain · 2026-04-12T01:47:08Z

Code review

No issues found. Checked for bugs and CLAUDE.md compliance.

🤖 Generated with Claude Code

_{- If this code review was useful, please react with 👍. Otherwise, react with 👎.}

- Create a fresh Live instance after each tool call instead of stop/start on the same instance, preventing the auto-refresh thread from overwriting stdout text with CURSOR_UP sequences - Respect NO_COLOR env var in query.py (consistent with chat.py) - Fix print("\n") → print() in chat.py finally block to avoid extra blank line when Live is active

Custom Rich Theme: bold headings without color, dark background for inline code, subtle link/list/blockquote colors. Shared via _make_rich_console() used by both chat and query.

- Headings: blue (#5fa0e0) - Code: yellow-ish on dark background - List bullets: green (#6ac0a0) - Bold: bright white, italic: light gray - Paragraph text: light gray for visibility

rejojer · 2026-04-12T16:00:48Z

Code review

Found 1 issue:

Duplicate text rendering after tool calls in both chat.py and query.py. The collected list accumulates all text deltas across the entire response, but Live instances are stopped and restarted around tool calls. When live.stop() is called, Rich commits the rendered Markdown to the terminal. A new Live is then created via _start_live(), but collected is never reset -- so the next live.update(Markdown("".join(collected))) re-renders all previously displayed text in the new Live region, causing visible duplication. This will happen any time the model produces text both before and after a tool call, which is a common agentic pattern.

chat.py -- collected is appended to at L277 and rendered at L280, but after the tool-call handler stops and restarts Live (L288-L302), the full collected content is re-rendered:

OpenKB/openkb/agent/chat.py

Lines 276 to 303 in 8cdc5a7

    
                           need_blank_before_text = False 
        
                       collected.append(text) 
        
                       last_was_text = True 
        
                       if live: 
        
                           live.update(Markdown("".join(collected), code_theme="monokai")) 
        
                       else: 
        
                           sys.stdout.write(text) 
        
                           sys.stdout.flush() 
        
           elif isinstance(event, RunItemStreamEvent): 
        
               item = event.item 
        
               if item.type == "tool_call_item": 
        
                   if last_was_text: 
        
                       if live: 
        
                           live.stop() 
        
                           live = None 
        
                       else: 
        
                           sys.stdout.write("\n") 
        
                           sys.stdout.flush() 
        
                       last_was_text = False 
        
                   raw = item.raw_item 
        
                   name = getattr(raw, "name", "?") 
        
                   args = getattr(raw, "arguments", "") or "" 
        
                   if live: 
        
                       live.stop() 
        
                       live = None 
        
                   _fmt(style, ("class:tool", _format_tool_line(name, args) + "\n")) 
        
                   live = _start_live() 
        
                   need_blank_before_text = True

query.py -- same pattern at L151-L167:

OpenKB/openkb/agent/query.py

Lines 150 to 168 in 8cdc5a7

    
                   if text: 
        
                       collected.append(text) 
        
                       if live: 
        
                           live.update(Markdown("".join(collected), code_theme="monokai")) 
        
                       else: 
        
                           sys.stdout.write(text) 
        
                           sys.stdout.flush() 
        
           elif isinstance(event, RunItemStreamEvent): 
        
               item = event.item 
        
               if item.type == "tool_call_item": 
        
                   raw = item.raw_item 
        
                   args = getattr(raw, "arguments", "{}") 
        
                   if live: 
        
                       live.stop() 
        
                       live = None 
        
                   sys.stdout.write(f"\n[tool call] {raw.name}({args})\n\n") 
        
                   sys.stdout.flush() 
        
                   live = _start_live() 
        
               elif item.type == "tool_call_output_item":

Fix: use a separate display buffer that resets when Live is restarted, while keeping collected intact for the final answer assembly at L309.

🤖 Generated with Claude Code

_{- If this code review was useful, please react with 👍. Otherwise, react with 👎.}

Two issues in the Live-based markdown streaming: 1. After a tool call, the new Live instance re-rendered the entire response so far because `collected` was never reset. Text before and after the tool call would be rendered together as one markdown block in the new Live region, with no separator between the two parts. Track the current segment separately from the full answer. 2. The tool_call_item handler started a new Live immediately after printing the tool line, which then had to be stopped in the text handler before the blank line. The empty Live start/stop plus the explicit `print()` produced two blank lines. Drop the premature start and let the next text delta create the new Live.

The previous approach used rich.markdown.Markdown with a hand-written Theme. Two problems: - Rich centers h1/h2 headings in a Panel, which looks out of place in chat output. - The Theme pinned a gray paragraph color (`#d0d0d0`) that overrode the terminal's default foreground for every line of assistant text. Replace with a renderer (openkb/agent/_markdown.py) that parses with markdown-it-py and maps each token to Rich primitives directly: Text for inline content, Syntax for code blocks, Group for block stacking. Plain text, bold, italic, and strikethrough keep the terminal's default color and only carry style attributes. Headings are left- aligned (bold, with h1 also italic + underline), list nesting uses decimal / letters / roman numerals by depth, blockquotes prefix non-blank lines with a dim vertical bar, tables render as pipes and dashes, and links emit OSC 8 hyperlinks where applicable. _make_rich_console loses the Theme (no reason to carry one now) and _make_markdown wraps the new renderer. query.py imports it from chat.py instead of constructing Markdown itself.

rejojer · 2026-04-13T10:08:58Z

Code review

Found 1 issue:

query.py tool_call handler eagerly restarts Live after printing the tool line (live = _start_live()), while chat.py uses a lazy-restart pattern (sets need_blank_before_text = True and starts a new Live only when the next text delta arrives). Commit 8708fc4 ("Fix streaming render bugs in chat and query") explicitly said "Drop the premature start and let the next text delta create the new Live" but only applied the fix to chat.py. The empty Live region between tool calls can reintroduce the blank-line/visual-noise symptom that commit was meant to fix.

OpenKB/openkb/agent/query.py

Lines 162 to 170 in dab7379

    
               args = getattr(raw, "arguments", "{}") 
        
               if live: 
        
                   live.stop() 
        
                   live = None 
        
               sys.stdout.write(f"\n[tool call] {raw.name}({args})\n\n") 
        
               sys.stdout.flush() 
        
               segment = [] 
        
               live = _start_live() 
        
           elif item.type == "tool_call_output_item":

Compare with the lazy pattern in chat.py:

OpenKB/openkb/agent/chat.py

Lines 248 to 284 in dab7379

    
                           if text: 
        
                               if need_blank_before_text: 
        
                                   if console is not None: 
        
                                       print() 
        
                                       segment = [] 
        
                                       live = _start_live() 
        
                                   else: 
        
                                       sys.stdout.write("\n") 
        
                                   need_blank_before_text = False 
        
                               collected.append(text) 
        
                               segment.append(text) 
        
                               last_was_text = True 
        
                               if live: 
        
                                   live.update(_make_markdown("".join(segment))) 
        
                               else: 
        
                                   sys.stdout.write(text) 
        
                                   sys.stdout.flush() 
        
                   elif isinstance(event, RunItemStreamEvent): 
        
                       item = event.item 
        
                       if item.type == "tool_call_item": 
        
                           if last_was_text: 
        
                               if live: 
        
                                   live.stop() 
        
                                   live = None 
        
                               else: 
        
                                   sys.stdout.write("\n") 
        
                                   sys.stdout.flush() 
        
                               last_was_text = False 
        
                           raw = item.raw_item 
        
                           name = getattr(raw, "name", "?") 
        
                           args = getattr(raw, "arguments", "") or "" 
        
                           if live: 
        
                               live.stop() 
        
                               live = None 
        
                           _fmt(style, ("class:tool", _format_tool_line(name, args) + "\n")) 
        
                           need_blank_before_text = True 
        
           finally:

Prior review comments

ANSI cursor overwrite after live.stop()/live.start() (query.py @ 5d2b4b4): resolved - a fresh Live instance is constructed each time, so position_cursor() starts from height 0 and cannot overwrite the prior tool-call text.
Duplicate text rendering from reusing collected across Live restarts (chat.py/query.py @ 8cdc5a7): resolved - a separate segment buffer is now maintained and reset at tool-call boundaries, while collected is preserved for final answer assembly.

🤖 Generated with Claude Code

_{- If this code review was useful, please react with 👍. Otherwise, react with 👎.}

- Drop eager Live restart after a tool call, mirroring the lazy pattern used in chat.py (commit 8708fc4 applied the fix only to chat.py) - Start Live inside the try block so a synchronous raise cannot leak it

- Preserve list item indent on multi-line paragraphs and block children - Render fenced/code blocks inside lists and blockquotes as plain text instead of leaking <rich.syntax.Syntax object at ...> repr - Preserve mailto link text when it differs from the email address

rejojer · 2026-04-13T10:39:14Z

Code review

Found 1 issue:

IndexError crashes the stream in _markdown.py _render_block fence branch when node.info is whitespace-only. The guard if node.info else "" is truthy for " ", then " ".strip().split()[0] hits [][0]. Confirmed: markdown-it-py emits info=" " for partial fence input like ``` followed by a space, which is exactly the intermediate state seen when streaming code blocks. Since render() runs on every streaming delta via live.update(_make_markdown(...)), this fires mid-stream whenever the model pads the opening backticks with whitespace before the language identifier. Fix: compute once and check, e.g. parts = (node.info or "").strip().split(); lang = parts[0] if parts else "".

OpenKB/openkb/agent/_markdown.py

Lines 54 to 58 in a9823fd

    
               return _render_inline_container(node) 
        
           if t == "fence": 
        
               lang = (node.info or "").strip().split()[0] if node.info else "" 
        
               return Syntax( 
        
                   node.content.rstrip("\n"),

Prior review comments

ANSI cursor overwrite after live.stop()/live.start() (query.py @ 5d2b4b4): resolved - fresh Live instance each time.
Duplicate text rendering from reusing collected across Live restarts (chat.py/query.py @ 8cdc5a7): resolved - separate segment buffer now used for rendering.
Eager Live restart divergence between query.py and chat.py (dab7379): resolved - query.py now uses lazy restart, stops Live and only recreates it when the next text delta arrives.

🤖 Generated with Claude Code

_{- If this code review was useful, please react with 👍. Otherwise, react with 👎.}

rejojer · 2026-04-13T10:47:15Z

Code review

No issues found. Checked for bugs and CLAUDE.md compliance.

Prior review comments

ANSI cursor overwrite after live.stop()/live.start() (query.py @ 5d2b4b4): resolved - fresh Live instance each time.
Duplicate text rendering from reusing collected across Live restarts (chat.py/query.py @ 8cdc5a7): resolved - separate segment buffer now used for rendering.
Eager Live restart divergence between query.py and chat.py (dab7379): resolved - query.py now uses lazy restart.
IndexError on whitespace fence info in _markdown.py (a9823fd): resolved - info_parts = (node.info or \"\").strip().split() with info_parts[0] if info_parts else \"\" guard at _markdown.py#L54-L58.

🤖 Generated with Claude Code

_{- If this code review was useful, please react with 👍. Otherwise, react with 👎.}

- Hoist MarkdownIt to a module-level singleton instead of rebuilding it on every call to render() - Re-render the Live region only when a text delta contains a newline, using the content up to the last newline as the visible snapshot. Flush the full segment once before stopping Live around tool calls and in the finally block so the trailing partial line is not lost

Replace `[tool call] name(args)` written via bare sys.stdout.write with chat's `_format_tool_line` (truncated to 78 chars, marker `·`) printed through `_fmt` with the `class:tool` style, so the tool-call line inherits the same color as in chat. Build the prompt-toolkit Style from chat's `_build_style` based on use_color.

When set, skips Rich Live + markdown rendering and writes the model's markdown source directly to stdout. Other styling is preserved: the chat prompt remains colored and the tool-call line keeps its `class:tool` style. This is distinct from --no-color / NO_COLOR, which strips all coloring.

rejojer · 2026-04-13T11:37:54Z

Code review

No issues found. Checked for bugs and CLAUDE.md compliance (no CLAUDE.md files present in the repo). Prior review comments on this PR all appear addressed by follow-up commits (5d2b4b4, 8708fc4, 80e7e7a, 88b61fc).

Generated with Claude Code

_{- If this code review was useful, please react with 👍. Otherwise, react with 👎.}

Avoid shadowing the new raw: bool parameter, matching query.py.

rejojer · 2026-04-13T12:00:38Z

Code review

No issues found. Checked for bugs and CLAUDE.md compliance. Prior review comments on this PR appear resolved in the current head (497ec2f).

🤖 Generated with Claude Code

_{- If this code review was useful, please react with 👍. Otherwise, react with 👎.}

Adopt the same last_was_text / need_blank_before_text state machine used in chat.py so tool-call separator blank lines are written lazily. This removes the extra blank line above tool lines and avoids a stray blank between consecutive tool calls or at the end of a turn.

rejojer · 2026-04-13T12:24:00Z

Code review

No issues found. Checked for bugs and CLAUDE.md compliance. Also verified that prior review comments (Live lifecycle reuse, fence IndexError, duplicate rendering, eager Live restart) have been resolved by follow-up commits.

🤖 Generated with Claude Code

_{- If this code review was useful, please react with 👍. Otherwise, react with 👎.}

KylinMountain added 2 commits April 12, 2026 09:40

Add rich Markdown rendering to single-shot query output

5d2b4b4

Apply the same rich Live + Markdown streaming render to `openkb query` as was added to chat. Falls back to plain text when stdout is not a tty.

KylinMountain force-pushed the feat/chat-rich-markdown branch from f206c75 to 5d2b4b4 Compare April 12, 2026 01:40

KylinMountain changed the title ~~Render Markdown in chat and query output using rich~~ feat: Render Markdown in chat and query output using rich Apr 12, 2026

KylinMountain requested a review from rejojer April 12, 2026 01:47

KylinMountain added 4 commits April 12, 2026 09:48

Apply Claude-Code-like theme to rich Markdown rendering

3e4d8d6

Custom Rich Theme: bold headings without color, dark background for inline code, subtle link/list/blockquote colors. Shared via _make_rich_console() used by both chat and query.

Add colors to Markdown theme for better readability

e6aeede

- Headings: blue (#5fa0e0) - Code: yellow-ish on dark background - List bullets: green (#6ac0a0) - Bold: bright white, italic: light gray - Paragraph text: light gray for visibility

Explicitly set code_theme=monokai for syntax highlighting in code blocks

8cdc5a7

rejojer added 2 commits April 13, 2026 17:55

rejojer added 2 commits April 13, 2026 18:29

Fix Live lifecycle bugs in query.py streaming

80e7e7a

- Drop eager Live restart after a tool call, mirroring the lazy pattern used in chat.py (commit 8708fc4 applied the fix only to chat.py) - Start Live inside the try block so a synchronous raise cannot leak it

rejojer force-pushed the feat/chat-rich-markdown branch from 61c49e6 to a9823fd Compare April 13, 2026 10:31

Fix IndexError on whitespace-only fence info during streaming

88b61fc

rejojer added 3 commits April 13, 2026 19:08

Rename raw to raw_item in chat tool-call branch

497ec2f

Avoid shadowing the new raw: bool parameter, matching query.py.

rejojer added 2 commits April 13, 2026 20:12

Preserve HTML in markdown renderer

3f1c676

rejojer changed the base branch from main to dev April 13, 2026 12:43

rejojer changed the base branch from dev to main April 13, 2026 12:45

rejojer approved these changes Apr 13, 2026

View reviewed changes

rejojer merged commit 9c75804 into main Apr 13, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Render Markdown in chat and query output using rich#22

feat: Render Markdown in chat and query output using rich#22
rejojer merged 17 commits intomainfrom
feat/chat-rich-markdown

KylinMountain commented Apr 12, 2026

Uh oh!

KylinMountain commented Apr 12, 2026 •

edited

Loading

Uh oh!

KylinMountain commented Apr 12, 2026

Uh oh!

KylinMountain commented Apr 12, 2026

Uh oh!

rejojer commented Apr 12, 2026

Uh oh!

rejojer commented Apr 13, 2026

Uh oh!

rejojer commented Apr 13, 2026

Uh oh!

rejojer commented Apr 13, 2026

Uh oh!

rejojer commented Apr 13, 2026

Uh oh!

rejojer commented Apr 13, 2026

Uh oh!

rejojer commented Apr 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

KylinMountain commented Apr 12, 2026

Summary

Test plan

Uh oh!

KylinMountain commented Apr 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

KylinMountain commented Apr 12, 2026

Code review

Uh oh!

KylinMountain commented Apr 12, 2026

Code review

Uh oh!

rejojer commented Apr 12, 2026

Code review

Uh oh!

rejojer commented Apr 13, 2026

Code review

Prior review comments

Uh oh!

rejojer commented Apr 13, 2026

Code review

Prior review comments

Uh oh!

rejojer commented Apr 13, 2026

Code review

Prior review comments

Uh oh!

rejojer commented Apr 13, 2026

Code review

Uh oh!

rejojer commented Apr 13, 2026

Code review

Uh oh!

rejojer commented Apr 13, 2026

Code review

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

KylinMountain commented Apr 12, 2026 •

edited

Loading